United States Patent [19] 

Lazarus et al. 



i 



US006134532A 
[li] Patent Number: 
[45] Date of Patent: 



6,134,532 
Oct 17, 2000 



[54] SYSTEM AND METHOD FOR OPTIMAL 
ADAPTIVE MATCHING OF USERS TO 
MOST RELEVANT ENTITY AND 
INFORMATION IN REAL-TIME 

[75] Inventors: Michael A. Lazarus, Del Mar; William 
R. Caid, San Diego; Richard S. Pugh; 
Bradley D. Kindig, both of Poway; 
Gerald S. Russell, San Diego; Kenneth 
B. Brawn, San Diego; Ted E. 
Dunning, San Diego; Joel L. Carieton, 
San Diego, all of Calif. 

[73] Assignee: Aptex Software, Inc., San Diego, Calif. 

[21] Appl. No.: 08/97L091 
[22] Filed: Nov. 14, 1997 

[51] Int. a. 7 - G06F 17/60 

[52] U.S. CI 705/14; 705/1; 705A4; 

705/26 

[58] Field of Search 705/14, 10, 1, 

705/27, 20, 26; 707/532 

[56] References Cited 

U.S. PATENT DOCUMENTS 

4,723,212 2/1988 Mindmm et al 705/14 

4,833,308 5/1989 Humble 235/383 

4,870,579 9/1989 Hey 705/27 

4,870,597 9/1989 Selri et a] 700/187 

4,970,681 11/1990 Bennett 707# 

4,996,642 2/1991 Hey 705/27 

5,331,544 7/1994 Lu et al 705/10 

5,353,218 10/1994 De Lapa et al 705/14 

5;459,306 10/1995 Stein el al 235/383 

5,515,270 5/1996 WeinblaU 705/14 

5,583,763 12/1996 Atcheson et al 707/3 

5,619,709 4/1997 Caid et al 707/532 

5,621,812 4/1997 Deaton et al 382/100 

OTHER PUBLICATIONS 

While Paper, "Building Customer Loyalty and Profitable 
1-to-l Customer Relationships with Net Perception's Grou- 
pLcns™ Recommendation Engine,"Copyright 1997, Net 
Perceptions, Inc. 



Firefly, "Building Relationships Intelligently: The Firefly 
Software Tools/'Copyright 1997, Firefly Channel Partners 
Seminar. 

Greening, "Improving Group Cohesion and Marketing 
Effciency with Like-Minded People, A White Paper on 
LikeMinds People Server 2.1" Copyright 1997, LikcMinds, 
Inc. 

Lashkari et al., "Collaborative Interface Agents," MIT Press, 
vol. 1, 1994, pp. 444-449. 

Shardanand et al., "Social Information Filtering: Algorithms 
for Automating 'Word of Mouth'," Human Factors in Com- 
puting Systems, CHI '95 Conference Proceedings, 1995, pp. 
210-217. 



Primary Exam iner — Em anue 1 Todd Voeltz 

Assistant Examiner — George D. Morgan 

Attorney, Agent, or Firm— Knobbe, Martens, Olson & Bear 

LLP 



[57] 



ABSTRACT 



Asystem and method for selecting and presenting personally 
targeted entities such as advertising, coupons, products and 
information content, based on tracking observed behavior on 
a user-by-user basis and utilizing an adaptive vector space 
representation for both information and behavior. The sys- 
tem matches users to entities in a manner that improves with 
increased operation and observation of user behavior. User 
behavior and entities (ads, coupons, products) and informa- 
tion (text) are all represented as content vectors in a unified 
vector space. The system is based on an information repre- 
sentation called content vectors that utilizes a constrained 
self organization learning technique to learn the relation- 
ships between symbols (typically words in unstructured 
text). Users and entities are each represented as content 
vectors. 

85 Claims, 23 Drawing Sheets 
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NAME 


TYPICAL 


VALUE RANGE 


DESCRIPTION 


threshold,^ 


0.5 


0.25-0.75 


Threshold by which query vector 
will be used instead of profile 
vector. 




0.1 


0.05-0.5 


Learning rate for profile update. 


Otod 


0.01 


0.005-0.1 


Learning rate for ad update. 




1 .ue— uo 


1.0e-05 - 
1 .0e-07 


Update rate for query universe 
estimate of the mean. 


£ profit* 


LOe-06 


1.0e-05 - 
1 .0e-07 


Update rate for profile universe 
estimate of the mean. 




2.5e-05 


1.0e-04 - 
1.0e-06 


Update rate for ad universe 
estimate of the mean. 


h profit* 


0.01 


0.001-0.5 


Forgetting factor for profiles. 


A„ 


0.01 


0.001-0.5 


Forgetting factors for ads. 




0 


Application 
dependent 


Mean of ads. 




0 


Application 
dependent 


Mean of profiles. 




0 


Application 
dependent 


Mean of queries. 



F/G. 20 
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SYSTEM AND METHOD FOR OPTIMAL 
ADAPTIVE MATCHING OF USERS TO 
MOST RELEVANT ENTITY AND 
INFORMATION IN REAL-TIME 

BACKGROUND OF THE INVENTION 

1. Field of Invention 
The invention relates to associating entities and informa- 



TABLE 1 -continued 



Method 



First Generation Ad Targeting Techniques. 
Disadvantage 



Browser type 



System type 

Service provider 

Geography/ 
location 

Site SIC Code 



tion with behavior. More particularly, the invention relates to 10 
a system and a process for targeting and delivering 
advertising, coupons, products, or informational content to 
users based upon observed behavior. 
2. Description of the Related Technology 
The widespread availability of the World Wide Web (web) ^ 
and Internet services has resulted in a unique set of adver- 
tising opportunities. Unlike conventional "broadcast" 
media, such as television and radio, the web is a "narrow- 
cast" medium that allows the user to have higher levels of 
control over the information they receive. Since users can 20 Compvi y size 
control the retrieval of information, the advertising tech- 
niques utilized in the conventional broadcast model has 
become less effective and can alienate potential customers as 
a result of the "shotgun" effect. The potential of selectively 
targeting advertisements on a user-by-user basis has been 25 Knowledge base 



unrealized due to the difficulty in performing meaningful 
targeting of customers. The current generation of web ad 
selection engines utilize a number of partially successful 
techniques to target customers. Typically, the effectiveness 
of these techniques is measured in terms of audience 30 
response rates. Audience response, also called 
"clickthrough", is evaluated by counting the number of users 
that click on a "banner advertisement" contained on a web 
page which is presented to the user. Clicking on the banner 
advertisement typically takes the user to the web site of the 35 
advertiser where additional information about a product or 
service is provided. In the current Internet advertising 
environment, clickthrough is the best measure of the effec- 
tiveness of advertising techniques. Consequently, the value 
of advertising is directly related to the effectiveness of the 40 
ad. Therefore, the maximization of clickthrough is of para- 
mount importance for most web sites for both practical and 
financial reasons. 

Some current banner advertising selection techniques are 
listed in the tables below. These techniques are divided into 45 
two classes. The first class of advertising selection 
techniques, shown in Table 1, use simplistic first generation 
techniques that are based only on static a priori information. 
The second class of advertising selection techniques, shown 
in Table 2, utilizes some of the techniques used in the more 50 
sophisticated second generation ad selection systems which 
may take into account some measure of user behavior, such 
as a user query, or make use of predisclosed user prefer- 
ences. However, these more sophisticated objectives often 
only complicate the problem. In Table 1 and Table 2 certain 
disadvantages of each method are given, however, the 
Tables are not meant to be all inclusive so not all disadvan- 
tages may be shown. 



Implies user behavior is related to usage domain 

Provides poor dickthrough 

Implies user behavior is related to usage browser 

type/version 

Provides poor dickthrough 

Implies user behavior is related to system type 

Provides poor clickthrough 

Implies user behavior is related to usage domain 

Provides poor dickthroogh 

Requires up-to-date knowledge base of IP address 

versus location 

Provides poor dickthrough 

Provides no significant user insig ht 

Requires up-to-date knowledge baso of SIC code 

versus IP address 

Provides somewhat improved user dickthrough 
Provides no significant user insight 
Requires up-to-date knowledge base of SIC code 
versus IP address 

Provides no significant user insight 
Implies user behavior is related to size 
of company 

Provides poor dickthrough 

Requires maintenance of knowledge base 

Costly to maintain 

Cannot incorporate observed user behavior 



techniques 



TABLE 2 



Method 



Second Generation Ad Targeting Techniques. 
Disadvantage 



User registration 


Requires user willingness to disclose information 


information 


Requires many judgments on products for 




effective operation 




Requires a substantial preexisting judgment 






Search key words 


Misses ads that are conceptually close but do not 




contain key word 




Restricts available inventory 




Requires careful choice of key terms 


Hand targeting of 


Very labor intensive; does not scale well 


ads to sites or 


Humans not good at manual targeting for large 


pages 


numbers of ads 


Hand analysis of 


Very labor intensive; does not scale well 


site or locations 


Humans not good at manual analysis of large 


in a site 


numbers of sites 


Random 


Poor response rates 


presentation 


Alienates users 


Dale/time 


Poor response rates 


Hand analyzed 


Very labor intensive 


sites 


High cost 



55 



TABLE 1 



First Generation Ad Targeting Techniques. 
Disadvantage 



Domain name of 
user browser 



Provides no significant user insight; Docs not 
account for user behavior 



In the current web environment, users utilize search 
services to find relevant or interesting information. These 
search services provide a potential focus for the identifica- 
tion of user behavior as denned by the searches they execute, 
the web pages they view and positions in the directory 
hierarchy they visit. However, existing banner advertising 
selection techniques fail to analyze this behavior when 
selecting an advertisement These search services provide an 
60 opportunity for presentation of user specific advertising. 
The conventional techniques shown in Table 1 and Table 
2, use hand coded associations and lists to perform customer 
"targeting." Certain techniques may use only the current 
user query as part of ad targeting. Each of the methods 
65 shown in Tables 1 and 2 may be associated with five general 
categories of advertisement selection techniques. These cat- 
egories include: 
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(i) rule -based categorization; 

(ii) keyword based ad selection; 

(iii) assigning ads to individual web pages or sites; 

(iv) assigning ads to branches in hierarchical organiza- 
tions of pages; and 

(v) collaborative filtering. 

A description of each category is provided below. 
Rule Based Ad Selection 

Rule based ad selection uses available information and 
rules to select an appropriate ad. This technique can be 
effective if large numbers of rules arc manually coded and 
the rule developer has a deep understanding of the problem 
domain. However, one problem with this system is that 
human intellectual effort is required to write or maintain the 
rules. The development of a rule base may be very expensive 
and time consuming. 

Additionally, the rules for advertisement selection are 
limited to available variables with discrete values and pro- 
vide "brittle" decision boundaries. These decision bound- 
aries arc typically binary and are frequently mishandled. 

Rule based ad selection requires extensive knowledge 
about the targeted operating domain. Even with computer- 
aided took, a knowledge engineer is required to develop the 
rules and administer the system. Furthermore, humans have 
demonstrated time and again, that they are poor at encoding 
rules. This observation is particularly true when large num- 
bers of variables are encompassed within the scope of the 
problem being modeled. 
Keyword Based Ad Selection 

In keyword based ad selection systems, the ads are 
selected on the basis of one or more user provided words. 
When an observed user behavior (typically a user issued 
query) contains a known keyword, one of the ads, which is 
manually associated with the keyword, is selected for dis- 
play. This technique provides good response rates for the 
keywords chosen. However, a major drawback with this 
approach is the system administrator must manually choose 
the keywords associated with each ad. This technique, based 
on intellectual effort and deep knowledge of the ad-specific 
domain, is time consuming and error prone. Additionally, the 
"inventory" of keywords at a site quickly becomes sold out. 
Lastly, with keyword based ad selection techniques, the ad 
selection process does not account for previous user behav- 
ior. The ad selector only uses a set of human-selected 
keywords in the current inventory based upon the current 
search query. Synonyms of user provided words are not 
automatically targeted without a thesaurus or synonym list. 
Assigning Ads To Individual Pages 

Ads are sometimes manually targeted to individual web 
pages. This method requires human intellectual effort to 
match an advertisement to a web page. However, such effort 
is usually prohibitive in large scale sites containing thou- 
sands of web pages. 

Assigning Ads to Branches with Hierarchies of Pages 

Ads are sometimes manually targeted to hierarchies or 
categorizations of pages called page ontologies. This tech- 
nique may provide effective performance if the human 
making the assignment of the ad to a branch of the hierarchy 
has a good feel for the content of the site and understanding 
of the intended viewer. However, human intellectual effort is 
required to select the appropriate branch or category for the 
ad. 

Assigning Ads To Web Sites 

Ads are sometimes manually targeted to all of the pages 
in a web site. However, as was seen with many of the other 
systems, human intellectual effort is required to select the 
appropriate ad for the web site. This method is inflexible in 
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that an advertiser cannot automatically target an ad to the 
best pages within a site. Typically, advertisers want to 
display different ads for each page in a site. Another problem 
with assigning ads to web sites is that the ads served to the 
5 user are not context dependent in that they fail to utilize user 
specific information. 
Collaborative Filtering Techniques 

Collaborative filtering (CF) techniques have been pro- 
posed for the problem of selectively targeting specific ads to 

10 web users. The CF approach requires large numbers of users 
to formally register with the system and make preference 
judgments about the quality of the ads, coupons and infor- 
mation content they receive. Under certain limited 
circumstances, this technique may provide some degree of 

is effective matching in small scale tests. However, for large 
tasks, such as found on the Internet, the algorithms become 
both computationally intractable and impractical. 
Additionally, the CF techniques perform best when the 
universe of entities are static rather than time evolving. In 

20 the Internet market, static content is the exception, not the 
norm. Furthermore, the utility of the CF approach is directly 
related to the numbers of users that have made preference 
judgments. Typically, users are reticent to spend a substan- 
tial amount of effort which is required to make these 

25 judgments both because of personal time limitations and 
concerns about privacy. As such, data acquisition becomes a 
major problem for the advertiser. 

CF solutions typically require a knowledge engineer to 
implement a system of sparse vectors of a very high 

30 dimension, where the dimension of the problem space is 
equal to the number of entities under consideration. 
However, this solution has been found to be computationally 
intractable. Another problem with CF techniques is that the 
ad selection software is examining for orthogonal relation- 

35 ships which requires a substantial number of statistically 
complicated steps. The ad selection software must find close 
neighbors in the vector space and suggesting coordinates not 
in common between the neighbors. 
The existing ad selection systems, particularly in coupon 

40 and print media advertising, fail to address the interactive 
nature of the Internet and electronic commerce. Advertisers 
need to be able to identify users of specific interests, track 
those interests over time and disseminate, in a highly selec- 
tive way, information, advertising, coupons and product 

45 offerings that will be of interest to the user. Additionally, 
advertisers need to track user interests and behavior in a 
real-time manner. 

Therefore, advertisers need a system which is sensitive to 
user behavior for targeting advertising, coupons, products 

50 and information content This system should enable the 
targeted marketing in real-time and to a granularity of an 
individual user as well as groups of users that have similar 
behavioral characteristics. 

ss SUMMARY OF THE INVENTION 

The present invention includes a system and method for 
delivering targeted entities, such as advertising, coupons, 
products and information content, to users based on tracking 
observed behavior. The system model is an adaptive vector 
eo space representation for entities, information and behavior. 
The system matches users to entities in a manner that 
improves with increased operation and observation of user 
behavior. 

The present invention uses an informational represente- 
es tion called entity or content vectors. The content vectors are 
a high dimensional, real vector representation that encodes 
contextual similarity. The system utilizes a constrained 
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self-organization learning algorithm to learn the relation- FIG. 4 is a block diagram illustrating the inputs and 

ships between symbols which are typically words in unstruc- outputs of the hash table creation process of the word 

tured text. This learning algorithm discovers relationships content vector learning module shown in FIG. 2. 

between words or symbols from a set of training examples plG. 5 is a block diagram illustrating the functions of the 

for a domain of interest and assigns a unique content vector 5 s t em generating process of the word content vector learning 

to each symbol. The resulting representation is such that module shown in FIG. 2. 

symbols that appear in a similar context will possess content pjQ 6 ^ a b]ock diagram illustrating the stem frequency 

vectors that point in similar directions in this high dim en- sUtistics calculation process of the word content vector 

sional content vector space. learning module shown in FIG. 2. 

Web pages, banner advertisements, coupons, and any 10 na 7 ^ a & mustratmg ^ different learning 

other textual or symbolic ^formation, are represented as a widows of the word content vector learning 

summary content vector by forming the normalized moduU , m nG 2 

weighted vector sum of the content vectors of the words „^ „ , .„ . . , . 

(symbols) in the page or description of the product or nG - « 15 i dupm illustrating the vector mampulation 

service. These summary content vectors have the property is P rocess for «*»P<»8 a stem vector by the vector of neigh- 

that pages that discuss similar information have summary stems for a convolution^ window shown in FIG. 7. 

content vectors that point in similar directions. FIG- 9 is a block diagram illustrating the inputs and 

The actions of each user are represented by profile vec- outputs of the entity vector creation process of the session 

tors. Typical user actions include, but are not limited to: compiler shown in FIG. 2. 

issuing queries requesting or reading pages, visiting Internet 20 FIG. 10a is a block diagram illustrating the inputs and 

sites, responding to ads, and redeeming coupons and pur- outputs of the process for creating the high speed files shown 

chasing products. A user's profile vector is configured to in FIG. 2. 

track observed behavior by using the content vector repre- FIG. 10b is a block diagram illustrating the inputs and 

senlation of the associated actions. For example, when the outputs of the process for generating the session control file 

user requests a page of information, the user's profile vector 25 shown in FIG. 2. 

is adjusted based upon the entity vector of the requested piG. 11 is a block diagram illustrating the inputs and 

P a S e - outputs of the process for transforming a page request into 

The user profile vector is utilized as the mechanism for an content vector which is performed by the real time ad 

selecting the best entity (e.g., ad, coupon, product or ^ sever module shown in FIG. 2. 

information) to present to the user. The user profile vector is pr G 12 is a block diagram illustrating the inputs and 

compared to each active entity vector, and the closest entity ou iputs of the process for transforming a query into a content 

content vector is selected for display to the user. vector ^ performed by the real time ad server module 

The present system also performs the optimal adaptive shown in FIG. 2. 

positioning of entity vectors. If a user is presented a banner 35 piG. 13 is a block diagram illustrating the functions of the 

ad and requests more information by clicking the mouse on ^ selection process performed by the real time ad server 

it, the entity vector for the ad is adjusted using the user's shown in FIG. 2. 

profile vector. Additionally, the user profile vector is n G . 14 is a block diagram iUustrating the functions of the 

adjusted using the ad content vector. This mutual adaptation fik veaof ^^on rocess of me profile vector u date 

allows optimum positioning of ads relative to users. ^ server ghown m HG 2 . 

In this system, the user profile vectors evolve through a nG 15 ^ a Wock milstI3Lling mc inputs md thc 

user's usage of the system. Clustering and classification QU of ^ modcI bum ^ rforaied by lhc 

techniques are applied to the adapted profile vector set to rcIalionship , discovery and analysis module shown in FIG. 

discover behavior patterns that are utilized for focused 2 

marketing campaigns. 45 F i G(16isablockdjagramiUustraangmefuKtionsoflhe 

Likewise, ad and coupon entity vectors evolve as a result ^ fa whkh me file vecU)r u ^ alg Mrw mo(Jule 

of user behavior These evolved vectors encode summary sbowQ m nG 2 ^ ^ fifc veclQr ^ 

characteristics of the users who clicked on the ad or demographic information. 

redeemed the coupon. These summary characteristics arc _„ . . , . ,. ... ^ . . c . , , 

used to modify product positioning and discover the char- 50 FIG ' " B \ b " d " 8 ?? 1 lUus f atin g * e 611,0110115 °* the 

acteristics of the users who are interested in thc product or P*™* 55 b ^ hl f ! he , enU % vecto _ r u P date *° m moduk 

shown m FIG. 2 adapts entity vectors. 

service. . . 

™ . . ,„ . .jjj . FIG, 18 is a flowchart illustrating the vector adaptation 

The system is fully automated and does not require any . . A . . , . r _ 

rules for operation and is based on self organizing neural P roccss ^ mc P rofilc ™ ctor ^ thc cntjt y 

, . . • Ajj-.- 11 .c . ■ „ vector udpdate server shown in FIG- 2. 

network techniques. Additionally, the system operates using 55 r 

tacit user observation and does not require explicit user FlGS - l9a and l9b are ^ e( l u f on flow diagrams for the 

judgments for correct operation. fixed rate learning algorithm performed by the profile vec- 
tors update module and the entity vector update module 

BRIEF DESCRIPTION OF THE DRAWINGS shown in FIG. 2. 

FIG. 1 is a functional block diagram illustrating a com- ^ FIG. 20 is a chart containing a summary of sample run 

puter environment that may be associated with the present time parameters for the fixed rate learning algorithm shown 

invention. in FIGS. 19a and 195. 

FIG. 2 is a functional block diagram illustrating the 

processing and data components of the present invention. DETAILED DESCRIPTION OF THE 

FIG. 3 is a block diagram illustrating the inputs and 65 PREFERRED EMBODIMENTS 

outputs of the preprocessing process of the word content The following presents a detailed description of certain 

vector learning module shown in FIG. 2. specific embodiments of the present invention. However, the 
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invention can be embodied in a multitude of different ways 
as defined and covered by the claims. In this description, 
reference is made to the drawings wherein like parts are 
designated with like numerals throughout. 

FIG. 1 is a block diagram illustrating a computer envi- 
ronment associated with the present invention. A client 
computer 100 has a monitor 102 and a processing unit 103. 
The processing unit 103 includes a data storage. 

The client computer 100 stores information that may be 
exported to other computing devices through a network 104. 
The network 104 may include any type of electronically 
connected group of computers including, for instance, the 
following networks: Internet, Intranet, Local Area Networks 
(LAN) or Wide Area Networks (WAN). In addition, the 
connectivity to the network may be, for example, remote 
modem, Ethernet (IEEE 802.3), Token Ring (IEEE 802.5), 
Fiber Distributed Dalalink Interface (FDDI) or Asynchro- 
nous Transfer Mode (ATM). Note that computing devices 
may be desktop, server, portable, hand-held, set-top, or any 
other desired type of configuration. 

When a remote user requests information from the net- 
work 104, the client computer 100 displays an advertisement 
with the requested information. Based on a request from the 
client computer 100, a server 106 may select an appropriate 
advertisement to display to the user based upon the user's 
observed behavior. 

In one implementation, the server 106 includes a gateway 
which is connected to a WAN 108. The WAN 108 has a 



10 



is 



20 



vectors. Thus informational items having similar meanings 
have closely aligned vectors, while informational items 
having dissimilar meanings have approximately orthogonal 
vectors. 

In the following description, the words "content vectors" 
and "entity vectors" are used synonomousjy. Both terms 
refer to a vector that is associated with certain content 
information, ad, coupon, solicitation, or product, for 
instance. 

The following description is divided into the sections: (1) 
System Component Overview, (2) Word Content Vector 
Learning Module, (3) Session Compiler Module, (4) Real- 
Time Ad Server Module, (5) Profile Vector Update Server 
Module, (6) Entity Vector Update Server Module, (7) Profile 
and Entity Adaptation Algorithms, (8) Calculation of Entity 
Inventory Weighting (Cost Model), and (9) Relationship 
Discovery and Analysis Processing Module. 

System Component Overview 

Beginning the discussion by reference to the bottom of 
FIG. 2, a word content vector learning module 200 creates 
a set of word content vectors from a domain relevant training 
corpus 202, a stop list file 204, and a phrase list file 206. The 
slop list file 204 contains a list of words, word fragments and 
phrases for the word content vector learning module 200 to 
ignore. The phrase list file 206 is a list of words that the word 
content vector learning module 200 groups together. The 
training corpus 202 is the set of informational objects used 



plurality of network servers 110. One of the network servers ^ to trato toe word vectors. The ranctior* of the training 
110 is connected to a LAN 112 comprising a plurality of 
computers 114. The system software that selects the adver- 
tisement may be located on one the network servers 110 or 
another computer in the network 104. In one embodiment of 
the invention, the advertisement selection software executes 
in part on a plurality of the network servers 110. In another 
embodiment of the invention, the advertisement selection 
software executes on a plurality of the computers 114 on the 
LAN 112. In yet another embodiment of the invention, the 



35 



corpus 202, the stop list file 204, and the phrase list file 206 
are further described below. 

When the word content learning process is complete, 
there is one word content vector in a word vector database 
208 for each word in the training corpus 202 that is not in 
the stop list 204. These word content vectors have the 
property that words that are used in a similar context have 
content vectors that point in similar directions in the high 
dimensional content vector space. The word content vectors 



advertisement selection software resides on the client com- ^ serve as the basis for the ad selection operations performed 



puter 100. It is important to understand that the advertise- 
ment selection system of the present invention may be 
hosted on any computing device so long as a communication 
pathway exists between the advertisement selection soft- 



by the present system during its operatioo. The word content 
vector learning process is performed once as part of the 
initialization process. However, these vectors can be 
updated if desired. Normally, the word content vectors are 



ware and the requester of the advertisement FIG. 2 is a 45 static during normal operation of the invention, since the 



functional block diagram illustrating the processing and data 
components of the present invention. The following descrip- 
tion focuses on the system's ability to deliver advertising, 
specifically banner advertising over the Internet. However, 



meaning of words in the English language evolves slowly 
over time. 

A session compiler module 210 takes a set of session 
control information, in the form of structured text from a 



the techniques utilized in this invention are general in nature 50 session description database 212, and transforms the text 



and can be applied to any type of entity or information. 
Furthermore, the network environment need not be the 
Internet as indicated previously. 
The present invention employs vector modeling of infor- 



into control commands for a real-time ad server (RTAS) 
module 214 in a session control file 215. Typical information 
processed during the compilation process includes the num- 
ber of active ads, the number of allowable impressions for 



mation so as to identify relationships among the informa- 55 each ad, start and stop times, and dates for snowing each ad. 



tional items or entity vectors. A process for making and 
using entity vectors is described in the U.S. Pat. No. 5,619, 
709 filed Nov. 21, 1995, entitled "SYSTEM AND 
METHOD OF CONTEXT VECTOR GENERATION AND 
RETRIEVAL" to Caid, et al. 

This entity vector generation scheme is designed to pro- 
duce content vectors. In an application to words, for 
instance, the scheme generates one vector per word, which 
represents the relative proximity of a meaning or content 
among words or documents in a quantitative and geometric 
manner. The system then creates an entity vector based upon 
the words in the entity and their associated word content 



The session compiler module 212 may be omitted from the 
system if such functionality is Dot needed. 

The present invention then computes an initial entity 
vector for each ad or entity in the system. In most cases, this 
60 entity vector is based on a textual description of the entity 
which is located in an entity description database 216. For 
instance, if the entity is an ad for a car, the words in the 
advertisement are used to determine the entity vector. The 
entity vector is computed as a normalized weighted sum of 
65 the word content vectors that are in the advertisement. A 
textual description of any item can be used as the initial 
condition for the entity vector. Since entity vectors can 
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evolve during system operation, the initial condition of the Hie EVUS module 228 performs a nearly identical action 

entity vector can be null, and no textual description of the as the PVUS module 224, but performs the operation on the 

entity is required for system operation. However, in normal entity vectors for advertisements. Each entity vector is 

operation, this evolution may require substantial time if stored in an entity vector database 230. For example, if the 

system usage rate is low and as a consequence, the preferred 5 uscr clicks on an ad, the entity vector for the user selected 

embodiment would start with any reasonable text descrip- ac j ls m0 ved a small step in the direction of the user's profile 

tion as an initial condition. vector 

In the present invention more ads can be sold in a A relalionshi ^ ^ ^ module 335 

thematic region of merest than m prev^s sy^cn^ smce aU proc 4ing actions on the contents of both the 

ads are selected on the basis of vector comparisons and not , n r r . . * Tr j t i_ 4 -» 1 . u 

exact matches, thus increasing the available inventory. For 10 ^ 5* V^i *7 

example, the provider of present invention could sell an J30 to .fnufy useful patterns .and trends. The RDAmodule 

adslotbasedupon the regionsof the vector hyperspace close 236 uul^s bo h clusteniigof the enuty vectors and profile 

to the words ifthe ad and not just a selected set of words. vectols t0 ldentlf y ^ behavior P attems ' 

The entity vector approach also allows administrators to . !t sbould bc ™ Xcd adaptation of the entity vectors 

effectively setup ad definitions without special skills or * not rc(nn «* for f^tcm operation. Indeed, when adapta- 

domain knowledge, thus improving overall system perfor- ^ 15 cnablcd > ll allows cnUtv vcctor of P roducl to 

mance without the requirement for domain experts. The Udrift " toward users who have received the ad. In some 

present invention automatically targets an ad to the best applications, information content delivery systems m 

pages within a site, not just the site as a whole. As a M particular this drift is not desirable. Clearly, the ^formation 

consequence of this feature, different ads may be associated ™ tcat ° f a ncws 15 deterministic and should not 

with the same page at a site. In addition, the present bascd on **» rcads the story. As such, the present 

invention automatically selects the most relevant pages invention has provisions to disable the adaptation of entity 

within a hierarchy or category for a particular ad. vectors. However, in many applications, such as Internet 

Alternatively, the system can be configured to serve the best „ advertising, the drift of the vectors is the mechanism 

ad independent of any ontology that might be present and whcrcbv thc svstcm discovers patterns and trends, 

bascd upon only on matching to observed user behavior. In the preferred embodiment described herein, the soft* 

In the present invention, the ads served to the user are ware programs are written in the C, C++ and TCL program- 
context dependent in that they depend on both the contents mjng language and run under the well-known UNIX oper- 
of the page, the current uscr action and the historical uscr 30 atiri g system. C, C++, and TCL are industry standard 
profile. Thc entity vector approach provides a solution which programming languages for which many commercial com- 
is compact and adaptive. The entity vector representation pikrs ca n be used to create executable code. However, the 
requires a much smaller amount of data for training as present invention may be run on a variety of conventional 
compared to the collaborative filtering approaches. hardware and software platforms. The software system 
Additionally, thc multidimensional entity vector represcnta- 35 described herein may be distributed across different sub- 
tion of thc present systcm leads to superior generalization systems running on different hardware platforms. The com- 
capabilitics as compared to the collaborative filtering ponents may communicate using messages transmitted 
approaches. using any network protocol, such as Transmission Control 

Referring again to FIG. 2, the real-time ad server (RTAS) Protocol/Internet Protocol (TCP/IP) and RCP. The specific 

module 214 is the engine which observes the user behavior, 40 P 1 ^ 01 ™ bost s these modules and files is unimportant to 

and based upon that behavior, selects the appropriate ad to me invention provided that the host computing environment 

present to the user computer 219 located at the top of the bas sufficient processing power. Additionally, each process- 

figure. The RTAS module 214 uses the current user action, m S 111(1/01 data component may be hosted on a separate 

the user's profile vector and the active set of entity vectors platform as long as paths of connectivity are provided via 

in selecting the appropriate advertisement. Thc RTAS mod- 45 any sufficient ly robust communication protocol, 

ule 214 uses vector proximity calculations such as inner The present invention includes six principle processing 

product to determine the optimal ad to display to the user. modules as follows: the word content vector learning mod- 

However, other distance metrics may also be employed If ule 200, the session compiler module 210, thc real-time ad 

the RTAS module 214 detects that a user has performed an server module 214, the entity vector update server module 

"adaptation event" on the client computer 219 such as 50 228. the profile vector update server module 224, and the 

issuing a query 217, requesting a page 218, purchasing a relationship discovery and analysis module 236. Below, the 

product or clicking on an ad 222, thc RTAS module 214 aforementioned components are described in further detail, 
sends an update message to thc a profile update server 

(PVUS) module 224 or an entity update server (EVUS) Word Content Vector Learning Module 

module 228. Thc RTAS module 214 utilizes a static content 55 The word content vector learning module 200 (shown in 

vector database 234 and a high speed file 232 to hold FIG. 2) is implemented as an application programming 

working values of the entity or ad vectors. Upon thc sclec- interface. The application programming interface may be 

tion of thc proper entity, the RTAS module 214 sends thc part of a computer program library. Advertisers use the 

selected ad to thc client computer 219. procedures provided by this library to create a stand alone 

The PVUS module 224 adapts the user profile vectors in 60 executable computer program capable of running on any 

response to a request from the RTAS module 214. A profile platform of sufficient processing power. This program per- 

vector for each user is stored in a profile vector database 226. forms the content vector learning process which in one 

This adaptation process moves the user profile vector in the embodiment, is further described in the Caid patent, 

direction of the entity vector associated with the adaptation Additionally, this program writes the output of entity vector 

event For instance, if the user clicks on an ad, the user 65 learning into data structures which are then utilized by the 

profile vector is moved a small step in the direction of the session compiler module 210 to create files optimized for 

entity vector for the ad. runtime efficiency. 
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The training corpus 202 which includes a large number of stemming process removes common suffixes from each of 

documents containing unstructured text or symbols is the words in the training corpus 202. 

selected and processed through the content vector learning FIG. 4 is a block diagram illustrating the inputs and 

algorithm. The resulting objects produced through the con- outputs of the process for creating a stem hash table 400. The 

teot vector learning process are then adapted for use by the 5 stem hash table 400 is created by a stem hash table creation 

other system components module 402. The stem hash table creation module 402 is a 

FIG. 3 is a block diagram illustrating the inputs and ^T^T*? °t ^ iS^I v ~ dor le ^/« 7>^ e 

outputs of a preprocessing module 300 of the word content For aUof the stems 302 m the utt^fik 30£ 

leading moduli 200 shown in FIG. 2. As part of the *J madc t m % stcm ^ table 400. Tins hash 

pre-protessing the word content vector learning performs io * blc also contains pomters for each stem to a unique data 

u * ii ■ * • element that consists of a stem text string, stem statistics, a 

the following functions: t ... j . . ^ ,.7- 

f stem content vector, and a correction content vector that is 

1. Forms input. used as temporary storage to accumulate the change vector 
Referring to FIG. 3, the requisite input files for the word to bc applicd to the slcm content vcctor as part of the 

content vector learning module 200 are assembled. The three learning process. 

input files are the training corpus 202, the stop list file 204 35 nG> 5 ^ a block ^gram illustrating the random number 

and the phrase list file 206. generating process for creating initial conditions for the stem 

In the situation where the symbols of interest are a series vectors. The initial conditions for each stem data item in the 

of words 301, the training corpus 202 includes a set of stem nash table 400 is created by a uniform pseudorandom 

unstructured free form text documents that cover the domain ^ aum ber generator (UPNG) module 500. The UPNG module 

of interest. For example, if the domain of interest is outdoor 500 generates an initial random content vector and stores 

sports such as hunting or fishing, an appropriate training mis vector m ^ stem c^ent vector. The UPNG module 

corpus might be 3 years of articles from Held and Stream 500 generates initial conditions for stem vectors through the 

magazine and the outdoor section of the local newspaper. ^ of a long period pseudorandom number generator which 

This text is typically a "flat" ASCII format with any type- ^ a flat distribution random number 502. Any random 

setting or formatting information removed. number generator can be used assuming the period of 

The stop list file 204 (shown in FIGS. 2 and 3) is repetition is sufficiently long (more than 100,000,000 

developed through a combination of human intellectual cycles). Using the flat distribution random number 502, a 

effort and conventional statistical techniques, that contains a transformer module 504, creates a Gaussian random variate 

list of words, word fragments and phrases for the word M 506 that has the property of zero mean and unit variance. An 

content vector learning module 200 to ignore. For the installation module 508 assigns the first Gaussian random 

content vector word learning, as with conventional statistical variate to the first element of the current stem content vector, 

information retrieval techniques, many words are meaning- a second random variate is then assigned to the second 

less. For instance, "AND", "NOT, "BUT', and "THE" are element of the stem content vector. This process of assigning 

words that convey little meaning in terms of determining the 35 un i quc Gaussian distributed random numbers is repeated 

themaUc content of a set of text. As a consequence, these all dements of all stem content vectors of all the stems 

terms should be ignored. The stop list file 204 is a flat ASCII m the stem hash tabic 400 have been populated, 

file that contains an exhaustive and enumerated list of words RG 6 ^ a bbck diagram illustrating the stem frequency 

to ignore with one entry per line. The typical length of this statistics calculation process of the content vector learning 

file for conventional applications is approximately 250-350 ^ module m FIG 2. A processing module 600 acquires 

of the statistically most frequently occurring words in the me stem f requency statistics from the intermediate stem file 

training corpus 202. 304 processing module 600 is a sub-component of the 

The phrase list file 206 (shown in FIGS. 2 and 3) contains word content vector learning module 200. The processing 
the list of word sequences that are to be treated as a single module 600 reads the intermediate stem file 304 and corn- 
symbol. Using this file, the word content vector learning 45 putes the number of occurrences of each stem 302 in each 
module 200 treats certain phrases as single symbols. For document in the intermediate stem file 304. The processing 
instance, "San Diego" should be considered as a single module 600 then stores these statistics in the stem statistic 
symbol The contents of the phrase list 206 are determined section of each data item 404 in the stem hash table 400. 
through intellectual effort, statistical techniques, linguistic FIG. 7 is a diagram of the different learning convolution^ 
processing or any combination thereof. The phrase list file 50 of the entity selection system shown in FIG. 2. A 
206 is not required for correct operation of the word content of cations to the initial conditions of the stem 
vector learning module 200, but the use of a meaningful cement vectors based on word co-occurrences are computed 
phrase list does improve system performance. The phrase for ^ ch of ^ ^ vectors ^ stem hash table 
list 206 is a flat ASCII file with one phrase per line in the file. 400 presen , system modines lhe stem vectors 

2. Performs word content vector learning. 55 based on the words that co-occur with a stem using a 
Once the training corpus 202, stop list file 204 and phase convolution al co-occurrence window. FIG. 7 shows an ini- 

list file 206 have been assembled, the word content vector tial source text 700. After some preprocessing, the source 

learning process is performed. The Caid patent provides a text 700 is transformed into the preprocessed text 702. The 

full disclosure on this approach, but a summary of the key system creates a convolutional window 704 to analyze a 

steps are repeated here for convenience. eo center stem and the center stem's neighbors. The window 

Referring to FIG. 3, the preprocessing is performed by a has a center and up to three words on cither side of the 

preprocessing module 300 which is part of the word content center, thus providing seven total words in the window. As 

vector learning module 200. The preprocessing module 300 can be seen in FIG. 7, the center of the window 704 is at the 

reads the training corpus 202, determines if the current word front of the document. As such, the window is one-sided, 

is on the phrase list file 206 or the stop list file 204, performs 65 The correction vector for the window 704, centered on the 

the stemming of the word, and writes a resulting valid stem stem "mao", consists of a vector function of the initial 

302 to an intermediate stem file 304 (shown in FIG. 3). The condition content vectors for "rebel", "group", and the 
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phrase " Sends ro Luminoso." Hie correction vector for a 
window 706, centered on stem "rebel", consists of a vector 
function of the initial condition content vectors for "mao", 
"group", "Sendero Luminoso" and "bomb." The window is 
moved one stem at a time and this process is continued for 
each stem in the p reprocessed text 702. At each update 
position, a correction vector is calculated and summed into 
an accumulated conection vector located in the data items 
404 of the stem hash table 400 (shown in FIG. 4). 



only two passes are required to achieve excellent learning 
results. The equations 1—4 are applied for each window 
position and the results are accumulated. At the end of each 
pass through all the stems in the intermediate stem file 304, 
the equations 5-9 are applied. The algorithm disclosed in the 
equations 5-9 computes the mean stem content vector, 
subtracts this from each normalized stem content vector and 
then re-normalizes the entire set of stem content vectors. 
This computation prevents the stem content vectors from 



FIG. 8 is a diagram illustrating a manipulation process for JO being localized to only one region of the hypersphcrc and 
calculating a stem vector by the vectors of its neighboring promotes better utilization of the vector space, 
stems. Each stem in the stem hash table 400 is modified by 
the vector of its neighboring stems. For each convolutional 
window, a vector 800 associated with the center stem is 
adapted by a vector 802, 804, and 806 which are associated 15 
with the stems of the center stem's closest neighbors. 

The following equations 1-4 provide the method for 
calculating the correction for each word in the convolutional 
window shown in FIG. 7. 



7f* = 



jftEV 



20 



M = 



1 

NSTEMS 



(5) 



(6) 



(7) 



(1) 



The value E tf represents the error or correction between the 
current target (center) T f and neighbor N^. A correction for 25 
Ty talring into account all current neighbors in window 
position is shown by equations 2 and 3. 



(*) 
(9) 



f-CUKSENT _ ^ACCUMULATED + gCVRXEttT 



(2) 



30 



(3) 



The variables shown in equations 2 and 3 are defined as 35 
follows: 

(i) i-summation dummy variable; 

(ii) C > ct/RR£wr -correction vector for stem j; 

(iii) Cf CCUMULATED ~munmg summed correction for 
stem j stored in stem hash table data elements; 40 

(iv) WS-window size in stems; and 

(v) ct^-a proximity constraint that limits the correcting 
effect according to the statistics of stem i and j where 



45 



e, = b + (l - B) 1 - 



l 



(4) 



50 



55 



Equation 4 is used to calculate the variable a^- for equa- 
tion 3. The variables shown in equation 4 are defined as 
follows: 

(i) ©-statistical importance of stem i in the training 
corpus; 

(ii) B -bounds control parameter determined empirically; 

(iii) NT) /-number of documents that contain stem i; and 

(iv) TND-total number of documents in the training 60 
corpus. 

Equations 5-9 included below provide a final update to 
the vectors. It should be noted that the algorithm described 
in the Caid patent utilizes a batched update process. As such, 
the algorithm accumulates corrections to the target (center) 65 
stem content vector. These corrections are made only at the 
end of a pass through the training corpus 202. Typically, 



The variables used in the equations 5 through 9 are defined 
as follows: 

(i) T > <J£,D -old target content vector (initial condition or 
previous pass); 

(ii) T^^-updated target content vector for current pass; 

(iii) Tf-lcarning rate; 

(iv) F^total number of instances of stem j in the training 
corpus; 

(v) C^^^^^-accumulated correction vector for 
stem i for this pass; 

(vi) M«mean stem vector; and 

(vii) N STEMS -total number of unique stems in the trig 
corpus. 

3. Writes output file. 

Once the training is complete, the resulting stem content 
vectors are written to the word vector database 208. 

Session Compiler Module 

FIG. 9 is a block diagram illustrating the inputs and 
outputs of the entity vector creation process of the session 
compiler module 210. In the present invention, the session 
compiler module 210 performs three functions: text to entity 
vector transformations, file format conversions, and session 
control file generation. Each of these functions is described 
below. 

1 . Text to entity vector transformations. 

To enable the selection of data or information, a text to 
entity vector representation transformation is required to 
generate the entity vectors. FIG. 9 illustrates the processing 
performed by the session compiler module 210 when it 
converts textual information into an entity vector. A prepro- 
cessing module 1100 reads the entity descriptions from the 
entity description database 216. The preprocessing module 
1100 is a sub-component of the session compiler module 
210 shown in FIG. 2. The preprocessing module 1100 
module is similar to the preprocessing performed as part of 
the preprocessing module 300 shown in FIG. 3. The pre- 
processing module 1100 determines if the current word read 
from the entity descriptions database 216 is on the stop list 
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file 204 or on the phrase list file 206. The valid stems 302 arc of entities, such as advertisements, that are eligible for 

then used by an entity vector formation module 1102 to selection to become active. The session control file 215 

generate static entity vectors which are then stored in the contains a control mechanism for the following attributes for 

static content vector database 234. each ad: 

Equation 10 is an example of the process performed by 5 Static or dynamic attribute control. An ad is either statically 

the session compiler module 210 shown in FIG. 2 to form «t t0 a fixcd location in the context vector space or 

entity vectors. The content vector for an entity is the allowed to dynamically adapt based upon user responses, 

normalized weighted sum of the symbols that comprise that Reset. If reset functionality is specified, the entity vector for 

entity. For those entities that are textual descriptions, the an existing ad is reset to its initial vector, 

description vector is the normalized weighted sum of the io Targeted impression count The targeted impression count 

stem content vectors of the stems in the description. determines the number of impressions that should be 

delivered for the ad. 

nstsms, (10) Start date. The start date determines the date and time that 

Z,J^ K the ad is to become eligible for selection. 

" ND t ) J 15 End date. The end date determines the date and time that the 

ad is to become ineligible for selection. 



ECVj = 



Z 



entity vector selection for the ad. Options for the initial 
" vector selection include: 



(TND\ Initial conditions. The advertiser can describe the initial 

20 (i) a vector based upon the ad and the aforementioned text 

The variables in equation 10 are defined: to vector conversion process; 

(0 ECV>the entity content vector for entity i; ( u ) a vector based upon the entity vector of a specified ad; 

(ii) SCV,— the stem content vector for stem j; ^ 

(iii) TND=the total number of documents in the training 25 a vector based u P° n 80 entitv vector contained in an 
corpus; external file. 

(ivjND.-the number of documents in the training corpus ^ s^ 00 compiler module 210 also establishes a 

that contain stem j- and connection with the EVUS module 228 (shown in FIG. 2) 

(v) NSTEMS.-the number of steins in the description of ^ * the "pository for all active entity information^ The 

ent -^ ■ initial entity vectors are calculated for the ads and the ad 

2. File format conversions identifiers, parameters, and vectors are transmitted to the 
FIG. 10a is a block diagram illustrating the process for module 225 for stora S e in the entitv vector database 

creating the high speed files 232 shown in FIG. 2. The 

session compiler module 210 performs the file format con- „ ,~ w . t 

. c -i » * .1. *■„ t • *l ,,- Real Time Ad Server Module 

versions to facilitate the entity selection process, since the 35 

data structure formats that are computationally efficient The RTAS module 214 provides an Application Program- 

during word content vector learning are inappropriate for ming for Interface (API) local or remote software applica- 

use by the RTAS module 214 during entity selection. The tions call to perform various ad management routines. The 

operational flexibility and complexity required for the word RTAS module 214 selects according to guidelines set in the 

content vector learning process is not required by the RTAS 40 session control file 215 an advertisement to be displayed to 

module 214, so substantial simplifications are made. As part a user. 

of this transformation, all of the stem content vectors are During ±& ^ se i ec t ioo processing, the RTAS module 214 

pre-scaled by their frequency of occurrence in the training performs the processing steps listed below, 

corpus 202. The stem content vectors are scaled by the same ^ Initialization 

factor shown in equation 10 without the final normalization 45 

step. Additionally, the low frequency stems (those that occur When ^ RTAS moduJe 214 receives an initialization 
less than four times in the training corpus) are removed from command, the RTAS module 214 reads the session control 
the file. The remaining stems represent the statistically most flle 215 ^ initializes internal control tables stored in 
significant portion of the overall stem population. The memory. The RTAS module 214 then requests the appro- 
session compiler module 210 then writes the resulting so P riate etnitv content vectors as specified by the contents of 
truncated and pre-scaled stem content vectors to the high me session control file 215. The RTAS module 214 requests 
speed files 232. fr° m lne entity vector database 230 via the EVUS module 

3. Session control file generation. 228 ^ emit y vectors for the active entities and stores them 
FIG. 10* is a block diagram illustrating the process for in a memory resident data structure within the RTAS module 

generating the session control file 215. The primary function 55 214 0nce initialization is complete, the RTAS module 214 

of the session compiler module 210 is the generation of a fe readv for reil -time operation, 

session control file 215 from operator created session 2. Determination of user behavior vector, 

descriptions contained in the session description file 212. When a user action message enters the system via the 

The session control file 215 contains the instructions in client API, the RTAS module 214 establishes the behavior 

machine readable format to guide the RTAS module 214 60 vector for that user. If the user is already known to the 

during the ad selection process. The session control file 215 system, the user id, provided by the user's web browser, is 

contains the information specified in the session description used as the key for a database retrieval. The RTAS module 

file 212 and a set of initial entity vectors. The session control 214 passes the user ID to the PVUS module 224. The PVUS 

file 215 has the ability to identify a group of ads eligible for module 224 then returns the profile vector associated with 

selection and to provide a set of backup entity vectors in 65 that user from the profile vector database 226. If the user is 

event the EVUS module 228 is not operational. The session unknown to the system, a new initialized vector is returned, 

information in the session description file 212 describes a set In either case, the RTAS module 214 has a valid user 
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behavior vector for subsequent operations. This vector is At a state 1602, the RTAS module 214 selects the closest 

stored within internal memory tables within the RTAS ad subject to inventory management constraints, 

module 214. Specifically, if the closest ad has already been presented 

3. Observation of user behavior. sufficient times to the user population, the RTAS module 214 
The RTAS module 214 also incorporates user behavior 5 P"** 11 * XCOIld dK * st matching selected. The 

into the user's profile vector. These behaviors include a web mventorymanagemern scheme can be arbitrarily complex 

page request 218 or a search query 217 as shown in FIG. 2. ™* modulc 214 has P™*^ i to interface to an 

The client computer 219 passes a behavior packet including c * crnal inventory management module that may be used to 

the observed behavior in the client API. The RTAS module adapuvely balance ad presentations subject to a acnes of 

214 uses the information contained in the user behavior 10 f*" 11 * 1 constraints. An inventory weighting module which 

packet to generate a behavior vector for the user action. 15 a ^-component of the RTAS module 214 performs all of 

FIG. U is a block diagram illustrating the inputs and * c mvcnt0 ^ f nt ^ ^"""J* "JiT? 

" f uidyam muiuauut, v closeness weights. The RTAS modulc 214 then measures 

outputs of the process for transforming a page request into ^ , irTu / . , \ «u\ 

. . . l * l ■ _r j u .u nT*c j. i and ranks each ad ID by score (state 1604). 

an content vector which is performed by the RTAS module 3 v ' 

214 shown in FIG. 2. If the observed user behavior includes 15 0nce * e closest entit y or ad has been «l«ted, subject to 

a page, navigation or hierarchy request 218, a page content the constraints described above, the RTAS module 214 

vector lookup module 1402 which is part of the RTAS P* 35 * 5 lhe ID of >*kcted ad via the API to the client 

module 214 retrieves the entity vector associated with the ID computer 219. The ID is used by the client computer 219 to 

of the requested page from the static content vector database relrieve ™d display the actual ad image to the user via the 

234. The RTAS module 214 augments the entity vector 20 web - 

module of the user requested page with the user ID to To improve performance, the RTAS module 214 caches 

provide a behavior vector. the set of entity vectors and parameters described by the 

FIG. 12 is a block diagram illustrating the inputs and active session control file 215. The RTAS module 214 

outputs of the process for transforming a query into a content obtains periodic updates from the EVUS module 228 for 

vector which is performed by the real- time ad server module changed entities. 

shown in FIG. 2. If a client computer 219 observes user 5. Observe user response to entity presentation, 

behavior such as a user query 217, the RTAS module 214 Once the best entity or ad is determined and the host 

transforms the query 217 into a content vector by using the system generates a web page that incorporates the selected 

memory image of the high speed file 232 to provide an entity x ad, the user response to the ad is monitored. If the user 

vector based upon the query terms. A preprocessing module requests more information on the product, as indicated by 

1502 which is part of the RTAS module 214, removes clicking on the entity or ad, this event is sent via the API to 

common words that are identified in the stop list file 204, mc RTAS module 214. However, all ad presentations need 

combines the words that are part of a phrase list file 206 and QOt result m a message to the RTAS 214 module, 

performs the stemming process for all of the terms in the 35 6 y of user vector . 

user query 217. The RTAS module 214 then creates a user . 

behavior vector by augmenting the result of the aforemen- ™ c RTAS „ mo ^ c 2U A ada P te mc ***«™ F° file 

tioned query 217 withlhe useTlD by a form entity content ™?° r . to reflect observed user *ctior^The observed user 

vectnr module 1504 behavior vector is then used to update the profile vector tor 

vector moauie idu» ^ ^ ^ RTAS 2U ^ ^ ]D ^ 

4, Selection of best entity to present. „ obscrved m bchavior ^ ^ ^ pvus modulc 2U for 
FIG. 13 is a block diagram lUustrating the functions ; of the modificatioQ of ^ historical user profile vector. The opera- 
ad selection process performed by the RTAS module 214 ^ of mc pyus modulc 224 ^ dcscribcd bclow m tcr 
shown in FIG. 2. If the chent computer 219 (shown in FIG. detail 

2) does not observe user behavior, the RTAS module 214 " 

uses historical user profile vector from the profile vector 45 7 ' "P** 6 of ad or enUt y vector 

database 226 as the basis for selection of the best entity to If the responds to the entity (ad, coupon or 

present. If the client computer 219 observes interactive solicitation), the entity vector and user ID are sent to the 

behavior, the RTAS module 214 uses the observed user EVUS module 228 for update (shown in FIG. 17). The 

behavior vector as the basis for selecting the best entity to Evus module 22S then adapts the entity vector toward the 

present. 50 observed behavior vector of the user. The operation of the 

The RTAS module 214 uses the historical user profile EVUS module 228 is dcscnbcd bclow in E rcater detail - 

vector or behavior vector as the basis of a vector comparison , T , « 

against all active ads as specified by the session control file Vcctor U P datc Scrvcr Modulc 

215. The contents of the session control files 215 .which arc The PVUS module 224 is responsible for the storage, 

loaded into memory by the RTAS module 214 during ss retrieval and updating of the user profile vectors that are 

initialization, are available for high speed searches. By contained in the profile vector database 224. 

placing the session control files in memory of the RTAS ^ profilc vcctors arc ^ basis tor a wide range 

module 214, a performance gam is realized. of proccssing capab iUties including: 

Referring again to FIG. 13, the RTAS module 214 per- ^ miDtit i oa of ^ted advertising, coupons, offers 

forms the optimum entity selection by performing dot prod- 60 ' informati on to the user; 
ucts of entity vectors with behavior and user vectors (state 

1600). The RTAS module 214 performs a search process by (») analysis of user behavior, 

computing a vector closeness metric between the vector to (h'O correlation of user behavior with advertising, product 

be compared and all active ad vectors. Although the RTAS and service descriptions; 

module 214 may use any vector proximity metric, the 65 (iv) inference of user characteristics based on cross- 
preferred embodiment of the invention uses a vector inner correlation of user behavior profiles with demographic 
product purchasing databases; 
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(v) tracking user behavior over time; FIG. 14 is a block diagram illustrating the profile vector 

(vi) predictive modeling of likely user behavior and adaptation process of the PVUS module 224 shown in FIG. 
optimization of information presentation based on 2. The PVUS module 224 contains several subcomponents 
these predictive models; and including a transform query to content vector (TQCV) 

(vii) accumulation of effectiveness and click-through sta- 5 module 1700, a transform action to content vector (TACV) 
tistics to optimize predictive models, provide manage- module 1704, a profile vector lookup (PVL) module 1706, 
ment reports and lay groundwork for increased ad and a profile vector adaptation (PVA) module 1702. If the 
revenues. user enters a query 217 to the system, the query string and 

In the preferred embodiment of the invention, the PVUS mc ID arc passed to a TQCV module 1700. The TQCV 

module 224 also incorporates external information, such as 10 mo aai e 1700 transforms the query string into a query 

credit card purchasing histories into the user profiles. This content vector in the same manner as the process shown in 

information is easUy incorporated, since the content vector pj a Wb. The resulting query content vector is passed to the 

representation accommodates the aggregation of heteroge- pyA module m2 fm ^ tatkm oithcvser profilc vcctor . 

neous symbolic information, such as web behavior and ^ pyA modulc m2 ^ ^ tf ^ ^ 

^S^^^^^^^L^ 15 dcsCTibcd tatow and m ^ GS - M« 1»- Aprofile 

The PVUS module 224 is implemented as one or more - , , , ( , , , _ A . . iL j . j 

executable computer programs tnat receive requests from ™*» ^ m ° dule 1704 , ™ lcs "P datcd 

clients over a network protocol such as TCP/IP and gencr- P rofilc vcctor 15 10 ^ profile vector database 226 for later 

ates responses to such requests. The PVUS module 224 has USC- 

one or more physical databases which are used to store the If mc ^ l cUcks on ad or purchases a product 222, the 

user profile content vectors and parameters which are 20 TACV module 1704 uses the ad or product description to 

indexed by unique user IDs. Multiple PVUS modules may adapt the user profile vector. This module 1704 transforms 

be in simultaneous operation to handle large numbers of the entity ID into a content vector via a table lookup in 

users. When multiple servers are used, they are addressed by memory within the PVUS module 224. In parallel, the user 

the clients as a single entity. The PVUS modules may be ID is used to lookup the user profile vector from the profile 

scaled across multiple hardware platforms to support a large 25 vector database 226. The PVA module 1702 inputs these two 

user population. This distribution is accomplished by mul- vectors into the adaptation algorithm described in FIGS. 19a 

tiplexing the user ID to select one of N physical servers. and 196. The profile vector database update module 1704 

Each server may further multiplex the request to one of M then writes updated profile vector to the profile vector 

physical databases. Multiplexing is achieved by generating database 226 for later use. 

a hash code from the user ID which results in essentially 30 4. Demographic predictions and tagging, 

random bit distributions in the hash code. There are many For many advertising and sales applications, it is advan- 

multiplexing hashing algorithms, that may be used. tageous to have an estimate of the demographic parameters 

However, the preferred embodiment uses the MD5 algo- for individuals as well as the entire user population. In the 

rithm. The multiplexor then uses a modulo N on the hash present system, although no user registration is performed, 

code to determine which of N children should handle the 35 advertisers can analyze the profiles vectors to retrieve user 

request. demographics. Advertisers can use these profile vectors to 

The PVUS module 224 performs the processing actions estimate the age, gender, household income (HI II), and other 

listed below. demographic parameters of a user. The present system has 

1. Creation of new user profile vectors. provisions to incorporate demographic models into its 
Each user in the system is assigned a unique user ID. In 40 operation. When using these demographic models, the 

the web environment this could be the user's Internet IP demographic parameters of the entire population of users are 
address or a host selected ID placed in the browser "cookie" estimated and used to augment the information in the profile 
file. If a request is made for a user ID that is not contained vector database 226. This estimate process can be performed 
in the profile vector database 226, the PVUS module 224 by both conventional statistical methods or via neural net- 
creates an entry for the new user and initializes the new user 45 work techniques. 

profile vector to a pseudorandom content vector which will FIG. 15 is a block diagram illustrating the inputs and the 

be later adjusted by the adaptation algorithm (shown in outputs of the model building process performed by the 

FIGS. 19a and 19£>). relationship, discovery and analysis module shown in FIG. 

2. Retrieval of user profile vectors and parameters. 2. Predictive models for the estimation of age, gender and 
The PVUS module 224 obtains a user profile vector by 50 HHI are constructed using a combination of previous 

using the unique user ID to determine a hash code. The entry observed behavior contained in a historical user actions 
in the hash table at that location contains a pointer to the database 226, ground truth information from user registra- 
information in the profile vector database 226. The PVUS tion on a different service or via a ground truth user 
module 224 uses the database pointer to perform the demographics database 1904, and profile vectors from the 
retrieval of the user profile vector. If the request for the 55 profile vector database 226. Using this information, a mod- 
profile vector is from the RTAS module 214, the PVUS eling module 1900 constructs a set of model parameters, 
module 226 passes the profile vector to the RTAS module These model parameters are stored in a model parameter 
214 for subsequent use. database 1906. Any statistically valid process can be used in 

3. Adaptation of user profile vectors. the modeling module 1900, including neural network 
The PVUS module 224 performs the adaptation of the 60 techniques, to derive the predictive model parameters. The 

user profile vectors based upon some observation about a typical outputs of the predictive model are age, gender and 

user's behavior. As the user traverses a web site utilizing the HHI. 

present invention, behaviors are observed including user FIG. 16 is a block diagram illustrating the process by 

requested queries, web pages and ads. Each observed behav- which the PVUS module 224 updates the profile vector 

ior is converted into a content vector and is also used to 65 database 226 with demographic information. For each user 

adapt a user's profile vector which is stored in the profile in the profile vector database, the PVL module 1706 

vector database 226. retrieves the user's associated profile vector from the profile 
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vector database 226. The PVL module 1706 passes this 
information, with the user ID and historical actions to a 
predictive model processing module 2000. The predictive 
model processing module 2000 uses the model parameters 
from a model parameter database 1906 in conjunction with 
the current user profile vector and previous observed behav- 
ior to form an estimate of a set of demographic parameters 
for the current user. This estimate is added to the information 
contained in the profile vector database 226 by a profile 
vector update module 1704. The augmented contents are 
used as input to the RTAS module 214 during ad selection 
and as part of the relationship discovery and analysis pro- 
cess. 

Entity Vector Update Server Module 

The EVUS module 228 is responsible for the storage, 
retrieval and updating of the entity profile vectors from the 
entity vector database 230. The EVUS module 228 is the 
main processing module in the system to read and to write 
the entity vector database 230. 

The EVUS module 228 consists of one or more execut- 
able computer programs that receive requests from clients 
over a network protocol (TCP/IP) and generates responses to 
such requests. The EVUS module 228 may have one or more 
physical databases which are used to efficiently store and 
index the user profile content vectors and parameters by 
unique entity IDs. 

During normal operation, the EVUS module 228 per- 
forms the following processing. 

1. Entity insertion and deletion. 

The EVUS module 228 inserts new entities or ads and 
removes old entities or ads from the entity vector database 
230 in response to commands from the session compiler 
module 210. When a new session is compiled, the session 
compiler module 210 sends update commands, new entity 
vectors and parameters to the EVUS module 228. 
Additionally, the session compiler module 210 sends com- 
mands to the EVUS module 228 to delete inactive entities 
from the entity vector database 230. 

2. Store and retrieve entity vectors and parameters. 
The EVUS module 228 stores and retrieves the current 

parameters and vectors for all the active entities in the 
system and retrieves the entity vector and associated param- 
eters for a given entity ID. During the initialization of the 
RTAS module 214, the RTAS module 214 sends a command 
to the EVUS module 228 requesting a copy of the most 
recent entity vectors and parameters to hold in the memory 
of the RTAS module 214. 

3. Entity profile vector adaptation. 

FIG. 17 is a block diagram illustrating the entity vector 
adaptation process of the EVUS module 228 shown in FIG. 
2. The EVUS module 228 performs the entity vector adap- 
tation in response to a message indicating that a user has 
responded to (clicked on) an entity. The EVUS module 228 
includes various sub-components including an entity vector 
adaptation module 2100, a transform query to content vector 
(TOCV) module 2102, an entity vector lookup module 2104, 
a profile vector lookup module 2106 and an entity vector 
database update module 2108. 

An adaptation event occurs when a user enters a query 
217 or clicks on an ad or performs an on-line purchase of a 
product 222. When the entity adaptation event occurs, the 
user action that resulted in the event, such as a user query 
217, the EVUS module 228 obtains from the RTAS module 
214 the entity ID for the selected ad and the user ID. 
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If the user requests a query 217, the TQCV module 2102 
transforms the actions into a query content vector, and 
passes the result to the entity vector adaptation module 2100. 
If the user event was a clickthrough or purchase 222, the 

5 entity vector lookup module 2104 retrieves the entity vector 
for the selected entity from the entity vector database 230. 
The entity vector lookup module 2104 passes the vector for 
the selected entity to the entity vector adaptation module 
2100. If the user ID was passed into the EVUS module 228, 

10 the profile vector lookup module 2106 retrieves the profile 
vector for the user from the profile vector database 226. The 
resulting content vector is then passed to the entity vector 
adaptation module 2100. 
After receiving the inputs from the TQCV module 2102, 

15 the entity vector lookup module 2104, and the profile vector 
lookup module 2106, the entity vector adaptation module 
2100 performs the equations shown in FIGS. 19Aand 19B. 
The entity vector adaptation module 2100 then passes the 
updated entity vector along with the entity ID and time 

20 stamp to the entity vector database update module 2108 
where the information is stored in the entity vector database 
230. 

4. Time updates. 

The EVUS module 228 generates a list of entity IDs 
25 whose vectors or parameters that have changed since a given 
moment in time. This information is used to refresh the 
memory image of the EVUS module 228 that is maintained 
by the RTAS module 214. When the refresh command 
containing a time stamp of the last update is sent from RTAS 
30 module 214, the EVUS module 228 sends to the RTAS 
module 214 the entity IDs, vectors parameters for those 
items that have changed and a new time stamp. This 
information is then saved in the RTAS internal memory for 
35 use in the entity selection process, 

5. Inventory management. 

The EVUS module 228 performs inventory management 
weighting across all the ads in the system. The EVUS 
module 228 uses a weight parameter to adjust each of the ad 
4Q selection scores. The weight parameter is determined by the 
ads targeted number of impressions and the time remaining 
for each ad. 

Profile and Entity Adaptation Algorithms 

45 FIG. 18 is a flowchart illustrating the vector adaptation 
process of the profile vector update server and the entity 
vector udpdate server upon the receipt of a user query 217 
(shown in FIG. 2). Both the PVUS module 224 and the 
EVUS module 228 apply a set of adaptation algorithms to 

50 allow the evolution of profile vectors and entity vectors as a 
result of user interaction with the system. 

Starting at a state 2250, the PVUS module 224 updates the 
profile vector based upon the current user query 217. Mov- 
ing to a decision state 2252, the profile vector of the user and 

ss the behavior vector for the search query is compared to the 
entity vectors and ranked according to this comparison to 
establish a score. The profile score of the user is compared 
against the behavior score for the search query to determine 
which score is larger. If the query score is larger, the PVUS 

60 module 224 proceeds to update the bias vectors (state 2254). 
The bias vectors arc used to make sure that during the 
continual mutual adaptation process that the vectors do not 
all end up with the same entity vector. Next, at a state 2258 
an advertisement is selected based upon the query score. 

65 Proceeding to a decision state 2260, the system checks 
whether a user clicked on the provided advertisement If the 
user did not select the advertisement the system returns to 
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the state 2250 to repeat the adaptation process upon future updating the ad vector with a user query after the user 

user queries. If the user selected an advertisement, the selected an advertisement. Similarly, the equations in block 

EVUS module 228 updates the selected ad vector with the 2280 are performed at the state 2264 during the update of the 

behavior vector for the user query 217 (state 2262). Next, at profile vector with the selected advertisement, 

a state 2264, the PVUS module 224 updates the profile 5 The equations in block 2282 (shown in FIG. 19a) are 

vector with the entity vector for the advertisement and performed at the state 2254 (shown in FIG. 18) to update the 

moves to a state 2274. bias vectors to insure that all of the vectors do not end up 

Referring again to the decision state 2252, if the profile with & c same multidimensional vector. The equations in a 

score is greater than the query score, the bias vectors are W° ck 2286 ^ performed at the state 2272 during the update 

updated in state 2265. Next, at a state 2266, the ad selected i° an ad vector with the profile vector. These equations are 

by the profile scoring is sent to the client 219. Proceeding to performed when the profile score of the user is greater than 

a decision state 2268, the system checks whether a user ^ query score. Similarly, the equations in a block 2288 

clicked on the provided advertisement. If the user did not (shown in FIG. 19b) are performed in the state 2270 (shown 

select the advertisement, the system returns to the state 2250 ^ HG. l g ) *e up<& lc of P rofilc vector with 

to repeat the adaptation process upon the receipt of future 15 a d vector. 

user queries 217. If the user selected an advertisement, the Th c equations in a block 2290 (shown in FIG. 196) are 

PVUS module 224 updates the profile vector with the entity performed during the update of the bias vectors after the 

vector for the advertisement (state 2270). The EVUS module profile score of the user is greater than the score for the users 

228 then updates the selected ad vector with the profile query at the state 2265 (shown in FIG. 18). 

vector for the user query 217. 20 

Moving from a state 2264 or a state 2272, at a state 2274 Model) 
the cost model associated with the selected advertisement is 

updated. The system then returns to the state 2250 to repeat The RTAS module 214 keeps track of the number of 

the adaptation process upon the receipt of future user queries impressions generated for each entity or ad and periodically 

217, 25 sends a message to the EVUS module 228 containing each 

The profiles of individual users and the corresponding i? 1 ^* 1 * 6 imp ^ DnS S eoerated 

current query are represented by the following two sets The RTAS module 214 uses the ad parameters specifying the 
shown in equations 11 and 12. number of impressions and the time each ad will 

remain eligible for serving to calculate inventory manage - 
30 ment weights. The weights are applied by the RTAS module 

P.jj-fpi'"* 0 ?J*oi (n) 214 when scoring the active ads. The RTAS module 214 

causes under-impressed ads to receive more impressions 
Q«irf-{qi (,-0 > - i l*'** 01 02) while suppressing impressions for over-impressed ads. 

User queries includes topical information which may be 35 Relationship Discovery and Analysis Processing 

represented by more than one content vector. Equation 13 Module 

shows that each ad is represented in a similar way to the _ r , . 

profile vectors P reseDt system has the ability to perform relationship 

discovery and analysis on the contents of the profile vector 
database 226 and the entity vector database 230 (shown in 

^(a/*^. . . a/"** 0 } (13) 40 FIG. 2). The results of the analysis process are listed in a 

series of reports that are presented to an analyst 240. These 

Equations 14 and 15 are applied during the decision state reports provide high value actionable information about the 

2252 shown in FIG. 15 when computing the scores. behavior pattern of the system users. 

45 The RDA module 236 takes the contents of the profile 

p_teon{P*ki> *\ - mux [prof^Vahic^u] (14) vector database 226 and the entity vector database 230 to 

produce the analysis reports 238 (shown in FIG. 2). A variety 

qjKon(Q aid , A t , ( ls ) of conventional statistical techniques or neural network 

.max [?, ■ af'VahKv^ x % ■ df* > threshold)) approaches are utilized to perform this analysis. 

iJMii 50 The RDA module 266 performs three analytical functions 

including unsupervised categorization, supervised categori- 

The Value parameter is computed by traditional cost ration and conventional statistics. Each of mese functions is 

model algorithms (not shown). described below. 

FIGS. 19a and 19i> are the equation flow diagrams for the 1- Unsupervised categorization, 

fixed rate learning algorithm performed by the profile vec- ss The RDA module 236 performs unsupervised categoriza- 

tors update module and the entity vector update module tion or clustering of the contents of the profile vector 

shown in FIG. 2. Unless otherwise noted, the adaptation database 225, the entity vector database 230 or both of these 

equations shown in FIGS. 19a and 19b are common to both modules. The RDA module 236 may use any clustering 

profile and entity content vector updates. The parameters algorithm as long as the algorithm can process real number 

used to control the adaptation process are shown in FIG. 20. so vectors. The result of the clustering is a set of cluster 

The following is a description of when the equations centroid vectors that represent regions of high density. These 

illustrated in FIGS. 19a and 196 are performed in relation to centroids are the same type of vectors that are used for all the 

the states illustrated in FIG. 18. The equations in a block other processing operations. Thus, the RDA system is able 

2276, 2284 (shown in FIG. 19a) are performed during the to find a set of word vectors that are closest in the content 

state 2250 (shown in FIG. 18) during the process of updating 65 vector space to the centroid vector. The words associated 

the profile vector with a query. The equations in a block with these vectors, and their closeness metrics, constitute the 

2278 are performed at the state 2262 during the process of key concepts of the centroid. The evolution of cluster 
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centroids over time provides trend information. The identi- is compQcd into a machine usable form. This results in the 

fication of word vectors close to these cluster centroids creation of a session control file 215 with information that 

provides a key word summary for each cluster, casing the controls the operation of the ad serving process. Finally, a 

analysis and interpretation task. Conventional statistical plurality of user profile vectors are initialized to NULL, 

analysis may also be used to investigate cluster contents. 5 Regular System Operation 

The variance of cluster membership is a reliable indication Durin S normal operation, an Internet user visits the web 

of cluster coherence and "unUy of focus" in terms of s , ilc «J d * e * established via login, or 

behavior. The cluster identified in the product vector data- ' hc .™ b hm ^ p ch ldcnWv * 

base 226 provides highly valuable information about the associated with a profile vector If the user is new, a null 

characteristics of theirs who purchase the products. This 10 ^J^^^^J^ t ^^J^ 7^1 

t . . r »l . . . . . • database. If the user is known to the system, the users 

result is a natural consequence of the mutual adaptation file vecl£)r fc retrieved from me m > vector dalabase , 

between the ad vectors and the user content vectors. The ^ meQ observes me ^ behavior Tf a ^ 

number of dusters found and me explanation of the clusters performs a search query 217, the user profile vector is 

are listed in the analysis reports 238 for inspection by the updated using the content vector of the current query. If the 

analyst 240. 15 user requests a web page to read 218, the user profile vector 

2. Supervised categorization. is modified by the content vector of the requested web page. 
The RDA module 236 also performs supervised catego- ^ updated profile vector is then used as the basis for 

rizalion of the contents of the profile vector database 226, selecting relevant ads to display to the user. Relevance is 

the entity vector database 230, or both of these modules. For determined by closeness of the user profile vector to entity 

supervised categorization, the RDA module 236 may 20 vectors stored m me ad vector database The closest entity 

employ any algorithm that can process real number vectors aS bci ? mo f "J™ 1 " 

tuipiwy <xuy <usvuuuii uiai wu V ivs*v> uuuii^i tvw » Jf me useT ffi presemec j an ad an( j tne user requests more 

The result of the categorization is a set of users and/or mformation of me of £ e ad b clickm £ on me ad 

entities that are members of each category. The count and ^ , he curren( ad eQ J ti vect0f ^ ^ ^ vect0f m 

membership list, by category arc listed in the analysis up d at ed. The user profile vector is moved or adapted a small 

reports 238 for inspection by the analyst 240. stcp m mc dircction of thc cntity vccton ^ CQtity vcctor ^ 

3. Conventional statistics. moved or adapted a small step in the direction of the profile 
Thc RDA module 236 applies conventional statistics to vector of the person who clicked on the ad. Thus, the system 

the contents of the profile vector database 226, the entity a U° ws to assume the characteristics of the users that 

vector database 230, or both of these modules. This analysis 30 Purchase them in a real-time adaptive manner. 

produces counts, count distributions, means and variances of Discovery and Analysis Mode 

the database contents as a whole or any subset which is . Dunng discovery and analysis mode at penodic intervals, 

selected by thc analyst 240. The results of the processing are SVSt ^ m ."ta?*** pcd^T^b^ da^ 

listed in the analysis reports 238 for inspection by thc analyst f ? . of contents ° f ^ th % e ? btv ^f" 

"V: J iv-Hwuo wo iv! ui^wtiuu uy au*ijoi database 230 and the user profile vector database 226 to 

3S discover useful and exploitable characteristics of user 

A summary of the operation of the system for the adver- behavior. The RDA module 236 performs vector clustering, 

tising scenario is as follows. reporting and provides summary statistical information of 

System initialization system effectiveness in terms of user clickthrough rate. The 

First, the word content vector learning module 200 creates behavior clusters discovered during analysis serves as the 

multiple content vectors from text input. This technique, 40 basis of advertising campaigns and provides valuable insight 

results in a set of word content vectors, one word content into user behavioral preferences. The behavior clusters can 

vector for each word found in the training vocabulary and be utilized as the initial conditions for entity vectors the 

not in the stop list 204. In the advertising scenario, the advertiser wants to target by group, rather than on an 

training texts are usually product descriptions and trade individual basis. 

magazines. For instance, if the targeted domain of operation 45 The system shown in FIG. 2 has two sets of adaptive 

is sporting goods, the training corpus would contain the components. First are the user profile vectors. Second are the 

product descriptions from sporting goods catalogs, the con- content vectors for each of the advertisements. Both sets of 

tents of several years of "Field and Stream" magazine and vectors influence each other based upon the actions of the 

books on outdoor life. The selection of the correct training users. Ads are updated in such a manner as to adapt toward 

corpus, oriented toward the domain of operation, such as 50 regions of high user interest, thus improving system effec- 

outdoor sports, gives the correct "spin" on the usage of tiveness and automatically discovering group preferences, 

words. In the aforementioned example, "bow" becomes User profile vectors adapt based on the observed actions of 

associated with "arrow" as opposed to "violin" or an action the user, thus providing an accurate and timely representa- 

performed on stage at the completion of a performance. tion of user preferences both individually and as a group. 

Next, the session compiler module 210 computes an 55 Since the sets of vectors are dependent on a user's actions 

initial set of entity vectors from descriptions of the products and evolve with time, they encode a wealth of information 

for sale. These descriptions might come from the product about the behavior of the user population. These content 

catalog or be typed in by a human operator. For instance, vector sets can be "mined" with the analysis toots to provide 

"Warm Goretex, down filled parka perfect for those cold valuable actionable insight concerning both individual, 

winter nights and days on the ski slopes" might serve as the eo group and whole population behavior, 

text initialization for the entity vector for a coat The entity While the above detailed description has shown, 

vector is a normalized weighted sum of the word vectors of described, and pointed out the fundamental novel features of 

the words in the description. The session compiler module the invention as applied to various embodiments, it will be 

210 creates an initial entity vector for each ad in the system. understood that various omissions and substitutions and 

Thc session description file 212 describes the guidelines 65 changes in the form and details of the system illustrated by 

for ad placement, such as display times authorized, sites, those skilled in the art, without departing from the intent of 

which ads to show and bow many impressions are available, the invention. 
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What is claimed is: 

1. A computerized system for associating an observed 
behavior with items, comprising: 

a converter capable of converting the observed behavior 
to a behavior vector, 5 

a profile adapter capable of modifying a profile vector 
with the behavior vector; and 

a comparator capable of comparing the modified profile 
vector to a plurality of entity vectors, the entity vectors 1Q 
indicative of the items, so as to identify at least one 
entity vector closely associated with the observed 
behavior. 

2. The computerized system as defined in claim 1, 
wherein the observed behavior is selected from the group J5 
consisting of: submitting a query to a web site, requesting a 
web page, purchasing a product, visiting a merchant, visiting 

a web site, inquiring about a product, watching a TV show, 
and watching a movie. 

3. The computerized system as defined in claim 1, ^ 
wherein at least one of the items is selected from the group 
consisting of: a coupon, an advertisement, a solicitation, 
information relating to a product, information relating to a 
set of services, a page, section or chapter of a book, a 
document, a newspaper article, a movie, a TV show, a web ^ 
site, and a textual material. 

4. The computerized system as defined in claim 1, 
wherein the computerized system is connected to a network, 
the network selected from the group consisting of: an 
intranet, a local area network, and the Internet. ^ 

5. The computerized system as defined in claim 1, 
wherein the converter includes a page content vector lookup 
module which identifies the behavior vector based upon a 
page identifier. 

6. The computerized system as defined in claim 1, 35 
wherein the observed behavior comprises a user query, and 
wherein the converter includes an entity content vector 
module for transforming the user query into behavior vec- 
tors based upon the component words of the user query. 

7. The computerized system as defined in claim 1, ^ 
wherein the profile adapter modifies the profile vector based 
upon at least one parameter selected from the group con- 
sisting of: a threshold by which a behavior vector will be 
used instead of the entity vector, a learning rate for a profile 
update, a leaning rate for an entity update, an update rate for 4S 
a query universe estimate of a mean, an update for a profile 
universe update of a mean, a forgetting factor for a set of 
profile vectors, a forgetting factor for a set of entity vectors, 

a mean of a set of entity vectors, a mean of a set profile 
vectors, and a mean of a set of behavior vectors. 5Q 

8. The computerized system as defined in claim 1, 
wherein the comparator includes a vector closeness deter- 
mination module to calculate the distance between the 
modified profile vector and any one of the entity vectors. 

9. A system for selecting advertisements in a computer 5S 
environment, comprising: 

a database of electronic advertisements; and 
an electronic advertisement management system, com- 
prising: 

a converter capable of converting an observed behavior 60 
of a user computing device in the computer envi- 
ronment to a behavior vector, 

a profile adapter capable of modifying a profile vector 
indicative of the user with the behavior vector, 

a comparater capable of comparing the modified profile 65 
vector to a plurality of entity vectors, the entity 
vectors indicative of the electronic advertisements, 



532 

28 

so as to identify at least one entity vector closely 
associated with the observed behavior, and 
a selector accessing the electronic database with the 
identified entity vector so as to select at least one 
electronic advertisement to communicate to the user 
computing device. 

10. The system as defined in claim 9, wherein the selector 
includes inventory management to allow selection of an 
entity that is under-selected according to the selection sched- 
ule and to inhibit the selection of an entity that is over- 
selected according to the selection schedule. 

11. The system as defined in claim 9, wherein the selector 
includes inventory management to allow selection of the 
entity vector based upon a presentation delivery schedule. 

12. The system as defined in claim 9, wherein the 
observed behavior is selected from the group consisting of: 
submitting a query to a web site, requesting a web page, 
purchasing a product, visiting a merchant, visiting a web 
site, inquiring about a product, watching a TV show, and 
watching a movie. 

13. The system as defined in claim 9, wherein the com- 
puter environment is connected to a network, the network 
selected from the group consisting of: an intranet, a local 
area network, and the Internet. 

14. The system as defined in claim 9, wherein the con- 
verter includes a page content vector lookup module which 
identifies the behavior vector based upon a page identifier. 

15. The system as defined in claim 9, wherein the 
observed behavior is selected from the group consisting of: 
a user query, a page view, or a purchase of a product. 

16. The system as defined in claim 9, wherein the profile 
adapter modifies the profile vector based upon at least one 
parameter selected from the group consisting of: a threshold 
by which a behavior vector will be used instead of the entity 
vector, a learning rate for a profile update, a leaning rate for 
an entity update, an update rate for a query universe estimate 
of a mean, an update for a profile universe update of a mean, 
a forgetting factor for a set of profile vectors, a forgetting 
factor for a set of entity vectors, a mean of a set of entity 
vectors, a mean of a set profile vectors, and a mean of a set 
of behavior vectors. 

17. The system as defined in claim 9, wherein the com- 
parater includes a vector closeness determination module to 
calculate the distance between the modified profile vector 
and any one of the entity vectors. 

18. A computerized system for adapting an entity vector, 
comprising: 

a converter capable of converting an observed behavior of 

a user into a behavior vector; 
a profile adapter capable of modifying a profile vector 

indicative of the user based on the behavior vector; and 
an entity adapter capable of modifying an entity vector 

indicative of an item based on the profile vector or the 

behavior vector. 

19. The system as defined in claim 18, wherein at least 
one of the items is selected from the group consisting of: a 
coupon, an advertisement, a solicitation, information relat- 
ing to a product, information relating to a set of services, a 
page, a section or a chapter of a book, a document, a 
newspaper article, a movie, a TV show, a web site, and a 
textual material. 

20. The system as defined in claim 18, wherein the 
observed behavior is selected from the group consisting of: 
submitting a query to a web site, requesting a web page, 
purchasing, a product, visiting a merchant, visiting a web 
site, inquiring about a product, watching a TV show, and 
watching a movie. 
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21. The system as defined in claim 18, wherein the a converter capable of converting an observed behavior of 
computerized system is connected to a network, the network a user into a behavior vector; and 

selected from the group consisting of: an intranet, a local an entity adapter capable of modifying an entity vector 

area network, and the Internet indicative of an item based on the behavior vector. 

22. The system as defined in claim 18, wherein the 5 32. The system as defined in claim 31, wherein observed 
converter includes a page content vector lookup module behavior is selected from the group consisting of: submitting 
which identifies the behavior vector based upon a page a query to a web site, requesting a web page, purchasing a 
identifier. product, visiting a merchant, visiting a web site, inquiring 

23. The system as defined in claim 18, wherein the about a product, watching a TV show, and watching a movie, 
observed behavior is selected from the group consisting of: 10 33- The system as defined in claim 31, wherein the 
a user query, a page view, or a purchase of a product. computerized system is connected to a network, the network 

24. The system as defined in claim 18, wherein the profile selected frora ^e group consisting of: an intranet, a local 
adapter modifies the profile vector based upon at least one area network, and the Internet 

parameter selected from the group consisting of: a threshold 34 • J** m sj*«jm as defined in claim 31, where in the 

bywhichabehaviorvwtorwmbiusedinsteadoftheentity 15 'FT ? t ^ k S ' 

v^alearmngrateforaprofileupdate.aleanungrateJ ^a^jj 8 Wl8V,nr **** b " ed UP ° D 1 

anentityupdalcanupdaterateforaqueryuniverseesumate P fg 1 ^ stem ^ defiQed iQ daim n wherein tfae 

of a mean, an update for a profile universe update of a mean, observed behavior fe sdectod from lfae ^ of: 

a forgetting factor for a set of profile vectors, a forgetung a ^ querVf a page or a purcaa se of a product, 

factor for a set of entity vectors, a mean of a set of entity 20 36 ^ ^ defined in claim 31, wherein the item 

vectors, a mean of a set profile vectors, and a mean of a set & selected from the group consisting of: a coupon, an 

of behavior vectors. advertisement, a solicitation, information relating to a 

23. The system as defined in claim 18, wherein the entity product, information relating to a set of services, a page, a 

adapter modifies the entity vector based upon at least one section or a chapter of a book, a document, a newspaper 

parameter selected from the group consisting of: a threshold 25 article, a movie, a TV show, a web site, and a textual 

by which a behavior vector will be used instead of the profile material. 

vector, a leaning rate for a profile update, a learning rate for 37. A system for selecting advertisements in a computer 
an entity update, an update rate for a query universe estimate environment, comprising: 
of a mean, an update for a profile universe update of a mean, a database of electronic advertisements; and 
a forgetting factor for a set of profile vectors, a forgetting 30 an electronic advertisement management system, corn- 
factor for a set of entity vectors, a mean of a set of entity prising: 

vectors, a mean of a set profile vectors, and a mean of a set a converter capable of converting an observed behavior 

of behavior vectors. of a user computing device in the computer envi- 

26. A system for generating a profile vector in a computer ronment to a behavior vector, 

environment, comprising: 35 a comparater capable of comparing the behavior vector 

a converter capable of converting a plurality of observed to a plurality of entity vectors, the entity vectors 

behaviors of a user into an associated plurality of indicative of the electronic advertisements, so as to 

behavior vectors; and identify at least one entity vector closely associated 

a profile adapter capable of repeatedly modifying a profile with the observed behavior, and 

vector indicative of the user based on the plurality of 40 a selector accessing the electronic database with the 

behavior vectors. identified entity vector so as to select at least one 

27. The system as defined in claim 26, wherein at least electronic advertisement to communicate to the user 
one of the plurality of observed user behaviors is selected computing device. 

from the group consisting of: submitting a query to a web 38. The system as defined in claim 37, wherein the 
site, requesting a web page, purchasing a product, visiting a 45 computer environment is connected to a network, the net- 
merchant, visiting a web site, inquiring about a product, work selected from the group consisting of: an intranet, a 
watching a TV show, and watching a movie. local area network, and the Internet. 

28. The system as defined in claim 26, wherein the 39. The system as defined in claim 37, wherein the 
computer environment is connected to a network, the net- converter includes a page content vector lookup module 
work selected from the group consisting of: an intranet, a 50 which identifies the behavior vector based upon a page 
local area network, and the Internet identifier. 

29. The system as defined in claim 26, wherein the 40. The system as defined in claim 37, wherein the 
observed behavior is selected from the group, consisting of: observed behavior is selected from the group consisting of: 
a user query, a page view, or a purchase of a product. a user query, a page view, or a purchase of a product. 

30. The system as defined in claim 26, wherein the profile 55 41. The system as defined in claim 37, wherein the 
adapter includes modifies the profile vector based upon at comparater includes a vector closeness determination mod- 
least one parameter selected from the group consisting of: a ule to calculate the distance between the behavior vector and 
threshold by which a behavior vector will be used instead of any one of the entity vectors. 

the profile vector, a learning rale for a profile update, a 42. The system as defined in claim 37, wherein the 
learning rate for an entity update, an update rate for a query 60 selector includes inventory management to allow selection 
universe estimate of a mean, an update for a profile universe of the entity vector of an entity being under-selected accord- 
update of a mean, a forgetting factor for a set of profile ing to a selection schedule and inhibit the selection of an 
vectors, a forgetting factor for a set of entity vectors, a mean entity that is over-selected according to the selection sched- 
of a set of entity vectors, a mean of a set profile vectors, and ule. 

a mean of a set of behavior vectors. 65 43. The system as defined in claim 37, wherein the 

31. A computerized system for adapting an entity vector, selector includes inventory management to allow selection 
comprising: of the entity based upon a presentation delivery schedule. 
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44. A method of associating an observed behavior with 54. The method as defined in claim 50, wherein the 
items on a computer including a data storage, comprising: electronic advertisement is communicated to the user cam- 
converting an observed behavior to a behavior vector, P^Z dcvicc via a network, the network selected from the 

.... . . . « ■ . , group consisting of an intranet, a local area network, and the 

modifying a profile vector with the behavior vector, and ^ terneL 

comparing the modified profile vector to a plurality of 5 „ The method ^ defincd m daim 50 wherein ^ 

entity vectors, the entity vectors indicative of the items, obscrved ^ behavior is selected from the group consisting 

so as to identify at least one entity vector closely o£ submitting a query to a web site, requesting a web page, 

associated with the observed behavior. purchasing a product, visiting a merchant, visiting a web 

45. The method as defined in claim 44, wherein the s i te( inquiring about a product, watching a TV show, and 
observed user behavior is selected from the group consisting 10 watching a movie. 

of: submitting a query to a web site, requesting a web page, 56. The method as defined in claim 50, wherein the 

purchasing a product, visiting a merchant, visiting a web converting includes identifying the behavior vector based 

site, inquiring about a product, watching a TV show, and upon a page identifier. 

watching a movie. 57. The method as defined in claim 50, wherein the 

46. The method as defined in claim 44, wherein at least 15 observed user behavior comprises a user query, and the 
one of the items is selected from the group consisting of: a converting includes transforming the user query into behav- 
coupon, an advertisement, a solicitation, information relat- i°r vectors based upon the component words of the user 
ing to a product, information relating to a set of services, a query. 

page, a section or a chapter of a book, a document, a . 58. A method for adapting an entity vector on a computer 

newspaper article, a movie, a TV show, a web site, and a 20 including a data storage, comprising: 

textual material converting an observed behavior of a user into a behavior 

47. The method as defined in claim 44, wherein the vector, 

converting step identifies the behavior vector based upon a modifying a profile vector indicative of the user based on 

page identifier. the behavior vector; and 

48. The method as defined in claim 44, wherein the 25 modifying an entity vector indicative of an item based on 
observed user behavior comprises a user query, and the P rofile vector or the behavior vector 
converting step transforms the user query into vectors based 59 - ^ method » defi ™d "» claim 58, wherein at least 
upon the component words of the user query. one of me items 18 ***** from me »° U P insisting of: a 

49. The method as defined in claim 44, wherein the cou P on > an advertisement, information relating to a 
comparing step calculates me distance between the modified 30 solicitation, informaUon relating to a product, a set of 
profile vector and any one of the entity vectors. t*™* 5 ' a P a S e ' a or a cha P ter of a book ' a 

50. A method of selecting advertisements in a computer document, a newspaper article, a movie, a TV show, a web 
environment, comprising: Slte ' and a textual material. 

... , . . c . . , t 60. The method as defined in claim 58, wherein the 

providing a database of electronic advertisements; ... . t . ^ . , , ' 
* . , , , , .35 modifying the profile vector includes modifying the profile 

converting an observed behavior of a user computing vector based upon at least one parame ter selected from the 

device in the computer environment to a behavior consisting of: a threshold by which a behavior vector 

vector modifying a profile vector indicative of the user will ^ ^ of me entf|y vector> a leaming rate for 

with the behavior vector; a profile update> a leaming fate fof m emity updat6) m 

comparing the modified profile vector to a plurality of 40 up date rate for a query universe estimate of a mean, an 

entity vectors, the entity vectors indicative of the elec- update f or a pt0 $i c universe update of a mean, a forgetting 

tronic advertisements, so as to identify at least one factor for a set 0 f profile vectors, a forgetting factor for a set 

entity vector closely associated with the observed D f enuty vectors, a mean of a set of entity vectors, a mean 

behavior; 0 f a ^ profile vectors, and a mean of a set of behavior 

accessing the electronic database with the identified entity 45 vectors. 

vector; and 61. The method as defined in claim 58, wherein the 

selecting at least one electronic advertisement to commu- modifying the entity vector includes modifying the profile 

nicate to the user computing device. vector based upon at least one parameter selected from the 

51. The method as defined in claim 50, wherein the group consisting of: a threshold by which a behavior vector 
selecting includes selecting an electronic advertisement that so will be used instead of the profile vector, a leaming rale for 
is under-selected according to a selection schedule and to a profile update, a learning rate for an entity update, an 
inhibit the selection of an electronic advertisement that is update rate for a query universe estimate of a mean, an 
over-selected according to the selection schedule. update for a profile universe update of a mean, a forgetting 

52. The method as defined in claim 50, wherein the factor for a set of profile vectors, a forgetting factor for a set 
selecting includes selecting the entity vector based upon a 55 of entity vectors, a mean of a set of entity vectors, a mean 
presentation delivery schedule. of a set profile vectors, and a mean of a set of behavior 

53. The method as defined in claim 50, wherein the vectors. 

modifying includes modifying the profile vector based upon 62. The method as defined in claim 58, wherein the 

at least one parameter selected from the group consisting of: computer is connected to a network, the network selected 

a threshold by which a behavior vector will be used instead 60 from the group consisting of: an intranet, a local area 

of the entity vector, a learning rate for a profile update, a network, and the Internet. 

learning rate for an entity update, an update rate for a query 63. The method as defined in claim 58, wherein the 

universe estimate of a mean, an update for a profile universe observed user behavior is selected from the group consisting 

update of a mean, a forgetting factor for a set of profile of: submitting a query to a web site, requesting a web page, 

vectors, a forgetting factor for a set of entity vectors, a mean 65 purchasing a product, visiting a merchant, visiting a web 

of a set of entity vectors, a mean of a set profile vectors, and site, inquiring about a product, watching a TV show, and 

a mean of a set of query vectors. watching a movie. 
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64. The method as defined id claim 58, wherein the converting includes transforming the user query into vectors 
converting includes identifying identifies the behavior vec- based upon the component words of the user query. 

tor based upoD a page identifier. 77. The method as defined in claim 71, wherein the 

65. The method as defined in claim 58, wherein the modifying step including modifying for the entity vector 
observed user behavior comprises a user query, and the 5 modifies the behavior vector based upoD at least one param- 
convcrting includes transforming the user query into vectors eter selected from the group consisting of: a threshold by 
based upon the component words of the user query. which a behavior vector will be used instead of the profile 

66. A method of generating a profile vector in a computer vcctoI) a Jcaming rMe for a profilc updatc> a lcanung ralc &r 
environment on a computer including a data storage, com- m enUty update m updaU , fate for a ^ estimate 
pnsing. io o ^ a mcan ^ m ^date for a profile universe update of a mean, 

converting a plurality of observed behaviors of a user into a forgetting f actor f or a set Q f pro fii e vectors, a forgetting 

an associated plurality of behavior vectors; and factQr for a M| of endty vectQrSj a mean of a x{ of entity 

repeatedly modifying a profile vector indicative of the vectors, a mean of a set profile vectors, and a mean of a set 

user based on the plurality of behavior vectors. 0 f behavior vectors. 

67. The method as defined in claim 66, wherein the 15 78. A method of selecting advertisements in a computer 
computer environment is connected to a network, the net- mc hiding a data storage, comprising: 

work selected from the group consisting of: an intranet, a ... , , . , ... 

local area network, and the Internet. providing a database of electronic advertisements; 

68. The method as defined in claim 66, wherein the converting an observed behavior of a user computing 
observed user behavior is selected from the group consisting 20 device in the computer to a behavior vector; 

of: submitting a query to a web site, requesting a web page, comparing the behavior vector to a plurality of entity 

purchasing a product, visiting a merchant, visiting a web vectors, the entity vectors indicative of the electronic 

site, inquiring about a product, watching a TV show, and advertisements, so as to identify at least one entity 

watching a movie. vector closely associated with the observed behavior; 

69. The method as defined in claim 66, wherein the 25 accessing the electronic database with the identified entity 
converting includes identifying the behavior vector based vector, and 

Up 2« a JS agC ^^ n ^f cr , ^ fr selecting at least one electronic advertisement to commu- 

70. The method as defined in claim 66, wherein the nica(c tQ (hc uscr computing device. 

observed user behavior comprises a user query, and the „ ^ method „ defiQed m daim ?8 ^ 

converting includes transform*^ the user query into vectors mdudcs sclccti an clcctronic advcrtisctncnt ^ 

based upon the component words of the user query. ^ accord in g to a selection schedule and 

71. A method of adapUng an entity vector on a computer mhibitin thc x)ed&m of an clcctronic advertisement that is 
mcluding a data storage, comprising: over-selected according to the selection schedule. 

converting an observed behavior of a user into a behavior 35 80. The method as defined in claim 78, wherein the 

vector, and selecting includes selecting the entity vector based upon a 

modifying an entity vector indicative of an item based on presentation delivery schedule, 

the behavior vector. 81. The method as defined in claim 78, wherein the 

72. Thc method as defined in claim 71, wherein at least comparing includes calculating thc distance between thc 
one of thc items arc selected from the group consisting of: 40 behavior vectors to any one of the entity vectors. 

a coupon, an advertisement, a solicitation, information rclat- 82. The method as defined in claim 78, wherein the at 

ing to a product, information relating to a set of services, a least one electronic advertisement is communicated to the 

page, a section or chapter of a book, a document, a news- user computing device via a network, the network selected 

paper article, a movie, a TV show, a web site, and a textual from the group consisting of: an intranet, a local area 

material. 45 network, and the Internet. 

73. The method as defined in claim 71, wherein the 83. The method as defined in claim 78, wherein the 
computer is connected to a network, the network selected observed user behavior is selected from the group consisting 
from thc group consisting of: an intranet, a local area of: submitting a query to a web site, requesting a web page, 
network, and the Internet. purchasing a product, visiting a merchant, visiting a web 

74. The method as defined in claim 71, wherein the 50 site, inquiring about a product, watching a TV show, and 
observed user behavior is selected from the group consisting watching a movie. 

of: submitting a query to a web site, requesting a web page, 84. The method as defined in claim 78, wherein the 

purchasing a product, visiting a merchant, visiting a web converting includes identifying the behavior vector based 

site, inquiring about a product, watching a TV show, and upon a page identifier. 

watching a movie. 55 85. The method as defined in claim 78, wherein the 

75. The method as defined in claim 71, wherein the observed user behavior comprises a user query, and the 
converting includes identifying the behavior vector based converting includes transforming the user query into vectors 
upon a page identifier. based upon the component words of the user query. 

76. The method as defined in claim 71, wherein the 

observed user behavior comprises a user query, and the * * * * * 
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